-
- 389 views
- 1 answers
- 0 votes
-
I have a dataframe df with the below schema (Spark 2.4) root |– segId: string (nullable = true) |– time: …
- 377 views
- 0 answers
- 0 votes
-
How can I cancel a long pyspark foreachPartition operation? For example I have my code that handles a very large …
- 384 views
- 1 answers
- 0 votes
-
So i have a list list = [11, 5, 7, 2, 18] and a RDD of a list RDD = …
- 320 views
- 0 answers
- 0 votes
-
I have defined a custom function in python to calculate class-wise auc scores in a one-vs-rest fashion. It takes true …
- 330 views
- 1 answers
- 0 votes
-
I’m trying to use pyspark to read a parquet file saved on my local machine, but I keep getting the …
- 345 views
- 0 answers
- 0 votes
-
data = sc.parallelize([1,2]) data.map(lambda x:x).collect() I can assure you that the code is proper since it ran correctly in the …
- 345 views
- 0 answers
- 0 votes
-
Below is the structure of .sh script which we schedule via Rundeck: kinit -kt ${keytab_file} ${principal_name} while [ $timeCounter -lt …
- 0 views
- 0 answers
- 0 votes
-
- 310 views
- 0 answers
- 0 votes
-
i want to apply lemmatization for dataframe column using pyspark running in databricks.Refer the images for error.
- 366 views
- 0 answers
- 0 votes